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Abstract 

We provide new infinitesimal characterizations for strong invariance of multifunctions in terms of 
Hamiltonian inequalities and tangent cones. In lieu of the standard local Lipschitzness assumption on 
the multifunction, we assume a new feedback realizability condition that can in particular be satisfied 
by control systems that are discontinuous in the state variable. Our realization condition is based 
on H. Sussmann's unique limiting property, and allows a more general class of feedback realizations 
than is allowed by the recent strong invariance characterizations [16]. We also give new nonsmooth 
monotonicity characterizations for control systems that may be discontinuous in the state. 
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1 Introduction 

The theory of flow invariance plays an important role in much of modern control theory and optimization 
(see [1, 9, 10, 15, 22, 23]). For a given set valued dynamics F evolving on R n and a subset S C R n , 
the theory provides necessary and sufficient conditions under which the pair (F, S) is strongly invariant, 
meaning, for each T > and each trajectory y : [0, T] — > R" of F starting at a point in S we have y(t) G S 
for all t G [0, T]. For the special case where F is locally Lipschitz and S C R" is closed, infinitesimal 
characterizations for strong invariance are well known. For example, if F : 1" =J R™ is locally Lipschitz 
and nonempty, compact, and convex valued with linear growth and S C R n is closed, then it is well 
known (cf. [9, Chapter 4]) that (F, S) is strongly invariant if and only if F(x) C Tg(x) for all ie5, 
where Tg denotes the Clarke tangent cone (cf. Section 2 and Appendix A for the relevant definitions). 
However, this cone characterization can fail if F is non-Lipschitz as illustrated in the following simple 
example: Take n = l,S = {0}, F(0) = [-1, +1], and F(x) = {-sign(x)} for x ± 0. Then Tg(0) = {0}, 
so F(0) % Tg(0). However, (F, S) is strongly invariant. This example is covered by the main sufficient 
conditions for strong invariance in [16, 18]. 

On the other hand, consider the controlled differential inclusion 

n 

x € G(x,a) :=Y[gi(x,<Xi) Di(x) (1) 
<=i 

where each factor §i : R" x A — > R : (x, a) i— > gi(x,a) is locally Lipschitz, A C R m is compact, 
on G A := {measurable [0,oo) — > A} for each i, a = (cti,(X2, . . . ,«„), and each of the multifunctions 
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Di : R" =4 M is Borel measurable. (Throughout this note, 11™= 1 Si = Si x ■ ■ ■ x S n for subsets S, CI and 
n"=i Si = ( s i> • ■ • ' s «) f° r P om ts Si £ R.) The dynamics (1) can be viewed as an uncontrolled differential 
inclusion x £ F(x) by taking F(x) — U{G(x, a) : a £ A n }. By definition, the trajectories of (1) are those 
absolutely continuous functions y : [0, T] — ► M" defined for some T > and inputs a, £ *4 that satisfy 
y(t) £ G(y(t),a(t)) for (Lebesgue) almost all (a. a.) i e [0,T]. The ZVs can be interpreted as set valued 
state dependent disturbance perturbations acting on the individual components of the locally Lipschitz 
dynamics g = (gi, g%, . . . , g n ). With this interpretation, the values t *—> (3i(t) £ Di(y(t)) assumed by 
the disturbances are unknown to the controller; one only knows that each (3i(t) takes some value in 
Di(y(t)) for each t. However, the mappings gi and D i} the inputs t i— » on{t) £ A, and the current state 
t 1 > y(t) can be measured. The dynamics (1) include the example from the previous paragraph by taking 
n = 1 and gi = 1. The objective is to find sufficient conditions in terms of the t/i's and IVs under 
which all trajectories of (1) starting in a given closed set SCI" remain in S, i.e., such that (G, S) is 
strongly invariant. Since the ZVs are not necessarily Lipschitz (or even continuous), the dynamics G 
may be discontinuous in the state, so the usual strong invariance criteria for locally Lipschitz systems 
(cf. [1, 9, 22]) do not apply. Moreover, the dynamics (1) arc not in general tractable by the strong 
invariance results from [16], even if the ZVs are singleton valued. For example, take n — 2, g\ = 92 = 1, 
Di(x) = {1}, D%{x) = {1} if X2 > 0, and Z?2(x) = {10} if xi < 0. In this case, if / were a feedback 
realization for the G-trajectory y(t) — (t,t) satisfying the requirements of [16] (cf. Section 3.3 for the 
relevant definitions), then (1, 1) = f(t, (t,t)) for all t > 0. Since / is continuous in the state variable, we 
can find 7 > such that f(t,x) £ [1/2, 3/2] 2 for all i > and ||x|| < 7. In particular if t > 0, ||x|| < 7, 
and X2 < 0, then f(t, x) £" cone{Di(x) x £>2(x)} = {A(l, 10) : A > 0}, so the cone requirement on / from 
[16] cannot be satisfied. We prove strong invariance results for (1) in Section 4. 

Strong invariance theorems are of great independent interest because they have been applied in many 
areas of nonlinear analysis and dynamical systems theory. For some applications, it suffices to have 
characterizations of weak invariance, which is the less restrictive requirement that for each point x in the 
constraint set 5*, there exists at least one trajectory y of the dynamics starting at x such that y(t) £ S 
for alH > 0. While weak invariance characterizations have been shown under very general assumptions 
on the dynamics (e.g., locally bounded convex values and closed graph), the standard results on strong 
invariance generally require locally Lipschitz multifunctions. Strong invariance theorems have important 
applications in uniqueness and regularity theory for solutions of Hamilton- Jacobi-Bellman equations, 
stability theory, differential games, monotone systems in biology, and elsewhere (cf. [1, 2, 3, 9, 10, 22]). 
On the other hand, it is well appreciated that many important dynamics such as (1) may be non- 
Lipschitz or even discontinuous in the state and so are beyond the scope of the usual strong invariance 
methods. Therefore, the development of conditions guaranteeing strong invariance under less restrictive 
assumptions on the dynamics is a problem that is of considerable ongoing research interest. 

This motivates the search for new infinitesimal characterizations for strong invariance for non- 
Lipschitz differential inclusions which is the focus of this note. (Here and in the sequel, "non-Lipschitz" 
means "not necessarily locally Lipschitz in the state variable".) Donchev, Rios, and Wolenski [12, 18] 
recently proved strong invariance characterizations under the somewhat less restrictive structural assump- 
tion of one sided Lipschitzness. A completely different approach was pursued by Krastanov, Malisoff, 
and Wolenski [16] who gave a new Hamiltonian sufficient condition for strong invariance for a class of 
feedback realizable differential inclusions (see Section 3.3 for the relevant definitions). The results in 
[16] do not require any of the usual structural assumptions on the dynamics that are generally needed 
in strong invariant systems theory, and therefore can be applied to a more general class of systems. 
However, [16] requires a cone condition on the feedback realizations that is not in general satisfied for 
the dynamics we consider below (see Section 3.3 for more discussions on [16]). 

In this note, we provide a nontrivial extension of [16] by proving strong invariance under an alternative 
feedback realizability condition that can in particular be satisfied by the general non-Lipschitz system 
(1) and other examples that are not tractable by known strong invariance results. Our condition is based 
on Malisoff 's coercive upper envelope approach from [17] and Sussmann's unique limiting condition from 
[21]. We express our invariance results in terms of tangent cones and Hamiltonian inequalities. Our 
approach has the additional advantage over [16] that it is preserved under "stacking" in the following 
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sense: If two dynamics yi G G\{y\,ai) and 2/2 G G2 (2/2, 0^2) satisfy our feedback condition, then so 
does the "stacked" dynamic (2)1,2/2) £ G\(yi,ai) x 6*2(2/2, a^)- The realizability condition in [16] is not 
preserved under "stacking" . Starting from this "stacking" property, our results lead to new infinitesimal 
monotonicity characterizations (see Section 5 below). Moreover, our results can still be applied even when 
the constraint set S is not necessarily a closed subset of R™ (cf. Section 3.2 below). While our main 
theorem can be shown using Zorn's Lemma, the proof we give below is constructive, and in particular 
leads to a new approach to building viable trajectories for Caratheodory dynamics; see Remark B.l. 

This note is organized as follows. In Section 2, we state our feedback realization property precisely and 
illustrate its applicability to dynamics such as (1) that are not tractable by the known strong invariance 
theory. In Section 3, we state our new necessary and sufficient conditions for strong invariance and 
we explain in detail how our conditions improve on the known results. We prove our main results in 
Section 4. Section 5 shows how our invariance characterizations lead to new nonsmooth monotonicity 
characterizations for systems with set valued state dependent non-Lipschitz disturbances. We review the 
relevant arguments from [16] and background from nonsmooth analysis in the appendices. 

2 Feedback Realizability Hypothesis and Examples 

Our main object of study is an autonomous differential inclusion x G F{x). This includes measurable 
controlled differential inclusions x G G(x,a) with control constraints a G A by taking F(x) := G(x,A). 
By a trajectory of x G F (x) on an interval [0,T] starting at a point x G R", we mean an absolutely 
continuous function y : [0,T] — ► R™ for which y(0) — x and y(t) G F(y(t)) for (Lebesgue) almost all 
(a.a.) t G [0, T]. We let Traj T (F, x) denote the set of all trajectories y : [0, T] — > M™ for F starting at 
x for each T > 0, and Traj(P,x) := U T >oTraj T (F, x) and Traj(F) := UxeK"Traj(.F , x). For a control 
system x G G(x, A), we let Traj T (G, a, x) denote the set of all trajectories y : [0, T] — ► R™ for G(-, a(t)) 
starting at x, for each T > and a G A := {measurable a : [0,T] — > A}. In that case, we also 
set Traj(G,a,x) = UT>oTraj T (G, a, x) and Traj(G,x) = U cte ^Traj(G, a, x). For n > 1 (respectively, 
n = 1), a mapping G defined on a Borel subset of M™ is said to be measurable provided G is Borel 
(respectively, Lebesgue) measurable. 

A multifunction F : R" =$ K" is said to have linear growth provided there exist positive constants 
Ci and c 2 such that \\v\\ < c\ + c 2 ||x|| for all v G ^(2;) and x G W 1 , where || • || denotes the Euclidean 
supremum norm. For any interval /, a function / : I x R" — > R" is said to have linear growth (on I) 
provided x 1— ► F(x) := {f(t,x) : t G /} has linear growth. For any subsets G, P C R™ and any constant 
ry G R, we set G + nP := {c + np : c G C,p G P}. Also, B„ := {x G M™ : < 1}. A mapping 
F : M™ =t R™ is said to be lower semicontinuous provided for each x G R™ and e > 0, there exists 6 > 
such that f(a;') + ei3„ D F(x) for all x' G x + SB n ; it is said to be closed (respectively, compact, convex, 
nonempty) valued provided F(x) is closed (respectively, compact, convex, nonempty) for each x G R™. 
We say that F is locally bounded provided F{nB n ) is bounded for each 77 > 0. Throughout this paper, we 
assume that all our mappings from R™ are nonempty valued. Also, int(G) (respectively, bd(G)) denotes 
the interior (respectively, boundary) of any subset G of a Euclidean space. The ith component of a 
mapping F into R™ is denoted by Fi for i = 1, 2, . . . , n. 

A continuous function lo : [0, 00) — > [0, 00) is called a modulus provided it is nondecrcasing with 
w(0) = 0. For each T > 0, we let C[0, T] denote the set of all functions / : [0, T] x M" -> R™ that satisfy 

(Gi) For each x G R", the map 1 1— > f(t,x) is measurable; 

(G 2 ) For each compact set K C R", there exists a modulus such that for all t G [0, T] and 

xi, x 2 G K, ||/(i,a;i) - /(*, ^2) || < ^f,K{\\xi - x 2 \\)\ and 
(G3) / has linear growth on [0,T]. 

This agrees with the definition of C[0,T] in [16]. Our main hypothesis is that each y G Traj(F) is also 
the unique (generalized) solution of an appropriate initial value problem x — f(t,x), x(0) = y(0) for a 
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feedback realization / £ C[0, T]. However, we allow a more general class of feedback realizations than is 
allowed in [16]. We present our feedback realization hypothesis next. 

We need the following additional definitions. We let co (resp., co) denote the convex hull (resp., closed 
convex hull). For each subset P in Euclidean space, we set cone{P} = U{r]P : r\ > 0} (written conc{p} 
when P — {p} is singleton) and we define the projections pr^jP} = {pi : 3p = (pi,P2, ■ ■ ■ ,Pn) £ P}, 
so Fi{x) = pr i {F(x)} for each i and mapping F into 1™. We define the component cone (ccone) by 
ccone{P} = {q £ R" : Bp £ P s.t. qi £ conejpi} Vi}. When P = {p} is singleton, we write this as 
cconejj?}. Notice that v £ ccone{P} is a less restrictive condition than v £ cone{P}. For example, 
(1, -1) £ ccone{(2, -1)} \ conc{(2, -1)}. Note that if P 1 , P 2 C R™ and v t £ ccone{P} for i = 1 and 2, 
then (v!,v 2 ) £ cconc{Pi x P 2 } C R 2 ™. Given F : R" =t R", x £ R n , and T > 0, we set 

C' F ([0,T],x) := {/ £ C[0,T] : 3 7 > s.t. f(t,x) £ cconc{P(x)} for a.a. t £ [0,7) and all x £ x + ~fB n }. 

Given W C R", we call a mapping P : R™ =4 R™ W -weakly zeroing provided: [x £ W and p £ F(x)] 
=> {(pi, 0, 0, . . . , 0), (0,P2, 0, . . . , 0), . . . , (0, 0, . . . , 0,p„)} C P(x). We simply say that P is weakly zeroing 
provided it is R™-weakly zeroing. Any P that is the product of one dimensional multi-functions Fi 
satisfying £ Fi(x) for all i and x is weakly zeroing, but weak zeroing is more general since it allows 
images F(x) such as {p £ R 2 : p\ > 0,j>2 > 0,^2 < 1 — Pi}- We assume the following condition: 

(U') For each x, T > 0, and y £ Traj T (P, x), there exists / £ C^([0, T], x) for which y is the unique 
solution of y(t) = f(t,y(t)) on [0, T] starting at x. Also, P is weakly zeroing. 

When the conditions in (U') hold, we call / a feedback realization for the trajectory y. The prime 
notation signifies that our condition is a variant of the realizability condition (U) from [16]; see Section 3.3 
for a comparison of conditions (U) and (If). Since £ F(x) for all x £ R™, our dynamics are weakly (but 
not necessarily strongly) invariant for any constraint set 5*. In our tangential characterizations of strong 
invariance, our weak zeroing requirement can be relaxed to the requirement that P be bd(S')-weakly 
zeroing for the closed constraint set S (see Section 3.1). Notice that feedback realizations / can depend 
on the initial value x, and the trajectory y. For example, if P is compact-convex valued and Lipschitz and 
weakly zeroing, then (see [16]) we can satisfy (V) using f(t,x) := W°iF(x){y(t)} f° r an y V ^ Traj(P), 
where proj^^jq} denotes the (unique) closest point in F(x) to q. The following non-Lipschitz examples 
show how Condition (U 1 ) can be satisfied without necessarily having f(t,x) £ F(x) for all x and a.a. t. 

Example 2.1. Choose n = 1, P(0) = [—1, +1], and F(x) = {— sign(x),0} for x 7^ 0. To verify Condition 
(£/'), let T > 0, x £ R, and y £ Traj T (P, x) be given. Note that (P, {0}) is strongly invariant. Therefore, 
either (i) y(t) starts at some i/0 and then moves toward at unit or zero speed or (ii) y(t) = 0. If 
x 7^ 0, then (W) is satisfied using f(t,x) = — sign(a;)l£>(t), where l.o(i) is defined to be when y(t) = 
or y(t) = 0, and 1 otherwise. Then f(t,x) £ cone{P(x)} for all t £ [0, T] and x £ x + (\x\/2)B 1 . If 
instead x = 0, then we choose f(t, x) = £ cone{P(x)} for all i £ [0, T] and a; £ R. 

Example 2.2. Consider the following dynamics evolving on R": 

n 

xeF{x)=\[L l {x)D l (x), (2) 

i=l 

where we assume the following for each i: 

(Pi) x 1 — * Li(x) C R is locally Lipschitz and compact valued. 

(P2) 2^ 1 — ► Di(x) C R is measurable, locally bounded, and closed valued. 

(P3) £ Di(x) for all x £ R™, and x ^ conc{Pi(x)} is constant. 

where Li(x)Di(x) := {XS : A £ Li(x),S £ Di(x)}. Note that under (P3), the only possible constant 
values for cone{Pi(x)} are {0}, R, [0, 00), or (— 00, 0]. In particular, for each x,y £ R™ and (3 £ Di(x), 
we have (3 £ cone{Pi(y)}. These hypotheses allow the systems (1) from the introduction by taking 
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Li(x) = gi{x,A) for all i. To check that (2) satisfies (W), let x G R™, T > 0, y G Traj T (F,x), and 
: R™ — > [0, 1] be any smooth (i.e., C°°) function that is identically one on y and compactly supported. 
Then y is also a trajectory of 

n 

G(x) := [] <t>{x) co{L t {x)} A(x). (3) 

i=l 

Since each component x i— ► </>(x)co{Li(x)} is Lipschitz and compact and convex valued, standard 
parametrization results (cf. [4, Section 9.7]) allow us to rewrite (3) as G(x) — n™=i 9i( x > &i)Di{x) 
for some Lipschitz function g = (g\, g%, . . . , g n ) where gi : R™ x B\ — > R for all i. Since y G Traj T (G, x), 
there are measurable functions a, : [0, T] — > £>i and : [0, T] — > M, with /^(t) G Di{y{t)) for a. a. 
t G [0,T], such that y is the unique solution of 

n 

/(t,a;):=JJft(*.ai(*))A(t) (4) 

on [0, T] starting at x. This follows from the (Generalized) Filippov Selection Theorem (see [22, p. 72]). 
The local boundedness requirement in (H 3 ) guarantees that the /3,'s are (essentially) bounded. Using 
the fact that conejcoD} = cone{D} for all subsets one can verify (V) using the choice (4) of /. 

Example 2.3. We next illustrate how dynamics satisfying (If) can be "stacked" to build new dynamics 
that again satisfy (V). Assume two dynamics F\,F2 : R™ -3 M™ satisfy condition (V). Consider the 
"stacked" dynamics F : K 2 ™ =$ M 2 " defined by F(x) = Fi(xi) x F 2 (x 2 ). We claim that F satisfies (U') 
in dimension 2n. To see why, let T > 0, x = (xi,x 2 ) G M™ x M™, and y G Traj T (F, x) be given. Write 
V = (l/i, 2/2) : [0, T] — > M™ x R". Since y 4 G Traj T (F,, x,) for 1 < i < 2, we can find G C' F .([0, T], x 4 ) such 
that j/i is the unique solution of the initial value problem x, = /^(t, Xj), x,(0) = x, on [0, T] for 1 < « < 2. 
The requirement ([/') is then satisfied for y using the feedback realization f(t,x) := Xi), / 2 (^,X2)). 

On the other hand, since we do not in general have f(t, x) G conc{i 7 '(x)}, the stacked dynamics may not 
satisfy the more restrictive requirement (U) from [16] (see Section 3.3). 

The preceding examples play an important role when we apply our results to monotone control 
systems in Section 5. In the next two sections, we state and prove our main strong invariance results. 
For the relevant background from nonsmooth analysis, see Appendix A. 

3 Statement and Discussion of Main Results 
3.1 Sufficient Conditions for Strong Invariance 

Let Hp : W 1 x R™ — > [— 00, +00] denote the (upper) Hamiltonian for our dynamics F defined by 
Hp(x,d) := sup{(v,d) : v G F(x)}. For any subset D C R™, we write Hp(x,D) < to mean that 
Hp(x, d) < for all d G D. By definition, this inequality holds vacuously if D = 0. 

Theorem 1. Let F : R" =4 R" satisfy (W) and ^ : R™ — > R be lower semicontinuous. If there exists an 
open set WCR" containing S := {x G R™ : ^(x) < 0} for which H F (x, dp^(x)) < for all x G U, then 
(F, S) is strongly invariant. 

The statement of Theorem 1 is the same as the main theorem in [16] except Theorem 1 replaces the 
realization assumption (U) from [16] with ([/') (see Section 3.3 for a comparison of these two assump- 
tions). We prove Theorem 1 below by constructing appropriate Euler polygonal arcs; see also Remark 
B.l for an alternative nonconstructive proof based on Zorn's Lemma. Condition ({/') allows us to apply 
our results to a broad class of examples that are not tractable by [16] or other known strong invariance 
results (see Section 2). Note that we require the Hamiltonian inequality in a neighborhood U of S. The 
result is not true in general if the Hamiltonian condition is placed only on S. For example, take n = 1, 
*(x) = x 2 , and F(x) = {0,1}. Then S = {0} and H F (0,d P ^(0)) = 0, but in this case (F,S) is not 
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strongly invariant. The non-Lipschitz system in Example 2.1 is covered by Theorem 1 if we choose the 
verification function $/(x) = x 2 . In that case, the Hamiltonian condition reads Hf{x,^>' (x)) = for all 
x G R, so our sufficient condition for strong invariance is satisfied. 

Theorem 1 contains the usual sufficient condition for strong invariance for an arbitrary closed set 
S C R™ by letting ^ be the characteristic function xs of S; that is, xs( x ) — if x G S and is 1 
otherwise. Then 9 P *(x) = {0} for all x g bd(S), and d P V(x) = Ng(x) for all x G bd(5). This implies: 

Corollary 3.1. Let F : R" =t R™ satisfy ([/') and S C R" be closed. If H F (x,Ng(x)) < for all 
x G bd(5), then (F, S) is strongly invariant. 

In fact, Corollary 3.1 is the special case of Theorem 1 using U = M. n , ^> — xs, an d 5 = 5. For the 
proof of Theorem 1, see Section 4. The proof shows that Theorem 1 remains true even if the weakly 
zeroing requirement on F is relaxed to: There exists an open set Q containing bd(5) such that F is 
Q- weakly zeroing. Moreover, Corollary 3.1 remains true even if this requirement is relaxed to requiring 
that F be bd(5)-weakly zeroing. 

3.2 Strong Invariance Characterizations 

The converse of Corollary 3.1 does not hold, as illustrated by Example 2.1. There (F, {0}) is strongly 
invariant but N^(0) = R, so the Hamiltonian condition in the corollary is not satisfied. This means 
that the converse of Theorem 1 does not hold. On the other hand, it is desirable to have a sufficient 
and necessary condition for strong invariance, that is, a strong invariance characterization. One would 
also hope to have such a characterization in the more general situation where a non-Lipschitz dynamic 
F evolves on an arbitrary open subset OCR™ and the constraint set S C R™ is a (relatively) closed 
subset of O but is not necessarily a closed subset of R"; i.e., S = T C\0 for some closed subset f CI". 
Characterizations of this kind play an important role in the analysis of monotone control systems (see 
Section 5 below or [1]). 

In this subsection, we provide such a characterization in terms of the Bouligand tangent cone Tg 
(see Appendix A for the relevant definitions) . Strong invariance of (F , 5) under relatively closed state 
constraints was treated in [1]. There it was assumed that F is locally Lipschitz. Here we consider a more 
general dynamic F : O x A R™ of the form 

n 

x G F(x,a) = Y[gi(x,a)Di(x), x G O, a e A (5) 

i=l 

where A is a compact subset of a Euclidean space, each gi : O x A — > R is locally Lipschitz, and 

(H4) Each Dj is convex valued, lower semicontinuous, and satisfies (H2)-(H^). 

Our assumptions imply that 09ih F(x, A) satisfies (U') (see Example 2.2). We prove: 

Theorem 2. Let OCR™ be open and S be a (relatively) closed subset of O. Let F : O x A R" be as 
above. Then (F, S) is strongly invariant if and only if F(x, A) C Tg(x) for all x G S. 

We defer the proof to Section 4 which also shows that the requirement that G A (x) for all i and x 
can be relaxed to: There exist compact sets Ni C A 2 C . . . C Nk C . . . such that 

(i) O = U k N k and 

(ii) G Di(x) for all i, all x G bd(5 n N k ), and all k G N, 
which imply that the pairs (F, S Ci Nk) are weakly invariant. 

Remark 3.2. The tangential condition from Theorem 2 remains sufficient for strong invariance of (F, S) 
if (H4) is relaxed to requiring x >—> F(x,A) to satisfy (U') 1 by the same proof. This gives a sufficient 
condition for strong invariance for the dynamics (5) with relatively closed state constraints if we merely 
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assume (H^-^Hz) without necessarily having convex valued or lower semicontinuous dynamics. On the 
other hand, Example 2.1 illustrates that the tangential condition is not necessary for strong invariancc 
without assuming (H4) even if O = R™. In that example, there are no controls a, and (F, {0}) is strongly 
invariant but [— 1,+1] = F(0) % T^ 0} = {0}. This does not contradict Theorem 2 since F is not lower 
semicontinuous, so (#4) is not satisfied. 

3.3 Relationship to Known Strong Invariance Results 

Theorem 1 applies to a more general class of multifunctions than the known strong invariancc results 
because it does not require the usual Lipschitz, one-sided Lipschitz, or other structural assumptions on 
F, nor does it require the more restrictive feedback realizability condition from [16]. The papers [6, 8] 
provide strong invariance results for locally Lipschitz dynamics (see also [9, Chapter 4]). For locally 
Lipschitz F, Clarke [6] showed that strong invariance of (F, S) is equivalent to F{x) C T§{x) for all 
x G S, where Tg denotes the Clarke tangent cone (cf. [9] or Appendix A for the relevant definitions). 
See [8] for Hilbert space versions and [22] for other strong invariance results for Lipschitz dynamics 
and nonautonomous versions. For strong invariance characterizations under somewhat more general 
structural conditions on F (e.g., dissipativity and one-sided Lipschitzness), see [11, 12, 18]. 

On the other hand, Theorem 1 docs not make any such structural assumptions on F and allows 
non-Lipschitz dynamics such as (2) that are intractable by the standard strong invariance results. In 
[16], strong invariance was shown assuming the following on the multifunction F: 

(U) For each x G R", T > 0, and y G Traj T (i^, x), there exists / G C F ([0, T], x) for which y is the 
unique solution of the initial value problem y(t) — f(t 7 y(t)), y(0) — x on [0, T]. 

where C F ([0,T],x) = {/ G C[0,T] : 3 7 > s.t. f(t,x) G conc{F(x)}Vx G x + >yB n k a.a. t G [0,T]} and 
C[0,T] is as defined in Section 2 above. An important difference between (U) and (V) is that (U) requires 
f(t,x) to be locally in the cone of F{x) rather than the larger ccone so Condition (U) from [16] is a more 
restrictive requirement than our Condition (U'). In other words, our condition f(t,x) G cconc{f (1)} 
from ([/') means there are weights oj\, CO2, ■ ■ ■ , u> n G [0, 00) (possibly depending on t and x) such that 
fi(t,x) G u>iFi(x) for all i while (U) makes the further restriction that u>i = u>2 = • • • = w„ (see Section 
1 for an example where (U') holds but these weights cannot be chosen to be equal). The main results of 
[16] are the same as our Theorem 1 and Corollary 3.1 except with (U 1 ) replaced by (U). 

Condition (U) has an advantage because it does not require zeroing, but the argument in the next 
section shows that Corollary 3.1 remains true even if this requirement is relaxed to requiring that F 
be bd(S')-weakly zeroing. Notice however that if F\ 1 F2 : R n =t R™ are two dynamics that satisfy 
(U), then it is not in general the case that the "stacked" dynamic F(x) := Fi(xi) x ^2(^2) satisfies 
(U). In particular, if x — (xi,x 2 ) G M™ x R™, and if also /i,/2 G C[0,T] and 7 > are such that 
fj(t,Xj) G conc{Fj(xj)} for all Xj G Xj + jB n , a.a. t G [0,T], and j = 1,2, then it is not generally the 
case that (fi(t, Xi), f2(t, x 2 )) G cone{i ;i i(xi) x F 2 (x2)}, although we do have 

(/i(t,xi),/ 2 (t,x 2 )) G ccone{Fi(xi) x F 2 (x 2 )} for all x = (xi, X2) G x + 7^ 2 „ and a.a. tG[0,T], 

so F satisfies (V) in dimension 2n. Therefore, condition (V) has the advantage that it is preserved 
under "stacking" . Using this "stacking" property, we prove nonsmooth monotonicity characterizations 
that extend the corresponding results from [1] (see Section 5). Theorem 2 extends the strong invariance 
characterization for locally Lipschitz controlled dynamics and relatively closed state constraints from 
[1, Theorem 4] by allowing non-Lipschitz disturbances Di. In particular, Theorem 2 gives a tangential 
characterization for strong invariance for non-Lipschitz dynamics defined on all of O — R"; our results 
are new even for this particular case. 
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4 Proof of Main Results 



4.1 Proof of Theorem 1 

Fix T > 0, x G bd(5), e > 0, and / e C' F ([0, T], x). By definition, we can then find 7 G (0, 1) such that 
fi(t,x) G cone{Fj(a;)} for all i, a. a. t G [0,7), and all x G a; + 7#„. We define the mollification f E by 
(30) in Appendix A below and let f £ .i denote its iih component for i = 1,2, ... ,n. By reducing 7 as 
necessary without relabelling, we can assume that x + ^B n C U. By also reducing T > 0, we also assume 
T G (0,7). For each i G {1, 2, . . . , n}, t G [0,T], a; G R", and r > 0, we then set 

n n 

G'l, [t, x,r} = H Rj , G) [t,x,r] = H G s f . [t, x, r] , g f [t, x, r] := 1 + sup { | \p\ | : p G G e f [t, x, r] } , 

where i£*- = {0} for all j ^ i and 

i2j = G^[t,a5,r] := co{/ M (i, y) : | |y - x| | < 1/r}. 

It follows that G^Ji, X, r] C g f[t,x,r]B n everywhere for each i. By our linear growth assumption (G3) 
from Section 2, the sets Gy i are compact. The following consequence of the Clarke-Ledyaev Mean Value 
Inequality (Theorem 5 from Appendix A) extends Claim 4.1 from [16] to our more general feedback 
realizations /: 

Claim 4.1. If x G R", f > 0, k G N, and h > are such that 

1 27 
< h < — - — — — and x + hg f [t, x, k/n]B n C x + — B n , (6) 

oz/c g j \t, X , K J Tl^ o 

then 

*(x + hv) < *(x) + \ (7) 

holds for some v G Gj[t,x,k/n}. 

Proof. Set Y\ = x + hGf ^t, x, 16k] which is compact and convex. We first find v\ G G^ [t, x, 16k] such 
that 

(a; + /i(«i,0,...,0)) <*(*) + A. (8) 
Suppose no such 1^ exists. Since ^ is lower semicontinuous and Yi is compact, we would then have 

(5 := < min - *(x). (9) 

nfc yen y 

Notice that x, 16fc] < <7/[t, x, k/n]. By (6), we can therefore find A G (0, 3^) satisfying 

x + hg f [t,x,16k]B n + \B n C x + /i ff/ [t, a;, fc/n]S n + XB n C x + 7# ri C W. (10) 

Next we apply Theorem 5 from Appendix A with the choices Y — Y\ and S defined by (9). It follows 
that there exist z G [x, Y{\ + XB n and ( G dp^(z) for which 

S < min (C, v — x) = min (C,hv), (11) 

yen v&G c fA [t,x,16kY 

where [x, Y\] denotes the closed convex hull of x and Y\. By (10), z G x + jB n C Also, (6) combined 
with the choice of A gives \\z — x\\ < hg/[t, x, 16k] + A < hgf[t, x, k/n] + A < — ^. Therefore 

f° A (t, z) := (f e>1 (t, z), 0, 0, . . . , 0) G G £ fA [t, x, 16k]. (12) 
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Since z G x + "fBn, we know that fi(s, z) G cone{Fi(z)} for a. a. s G [0, T]. Since F is weakly zeroing, 
this gives (/i(s, z), 0, 0, . . . , 0) £ cone{F(z)} for a. a. s G [0, T]. Since z eU, our Hamiltonian hypothesis 
then gives 

(C,(/i(«,«),0,0,...,0)><0 for a.a. a € [0,T\. (13) 
Therefore, (11)-(13) give the contradiction 

< S < h(C,f°M,z)) = h f V e(t-a)(C,(fi{8,z),0,0,...,0))da < 0, (14) 



so there must exist vi G [£, x, 16k] C G^ [£, x, k/n] C gf[t, x, k/n]B\ satisfying (8). Assume n > 1. 

Set v° = (u!,0,0,.. .,0) € K". Then (a; + to?) + to^[£, x, fc/n] ({0} x S„_i) C x + hg f [t, x, k/n]B n , so 
(10) gives (x + hv°) + hgf[t, x, k/n]({0} x S„_i) + A<B„ C x + jB n . Moreover, 

G% [t, x + hvl, 16k] C G% [t, x, k/n], (15) 

because if \\y - (x + to?)|| < j^, then (6) gives \\y - x\\ < + h\ Vl \ < + ^ < ^ < 
which gives (15). We can therefore apply the preceding argument with x replaced by x + hv°, and with 
Y\ replaced by the new compact convex set 

Y 2 := (x + hvl) + hG s ft2 [t,x + hvl, 16k] C (x + hvl) + hg f [t,x,k/n]({0} x S„_i) 

to find z G [x + to?,Y 2 ] +AB„ such that f° t2 {t,z) := (0, / £ , 2 (£, z), 0, 0, . . . , 0) G G^ 2 [£, x + to?, 16fc] and 
v 2 G G^ 2 [£, x + to?, 16k] C G^ 2 [£, x, fc/n] such that 

*(x + to? + /i(0, v 2 , 0, 0, . . . , 0)) < *(x + hvl) + ~~T- (16) 

Next we argue by induction. Proceeding inductively, we apply the same argument but with x replaced 
by Xi-i := x + h{v\,v 2 , • ■ • , Ui-i, 0, 0, . . . , 0) and with the set Yi replaced by the new compact convex set 
Yi := Xi_i + /iGj j£, Xj_i, 16fc]. Since w r G G^ r [£, x,k/n] for all r < i, (6) implies that ft|u r | < ^ for all 
r < i, so the proof of (15) shows G e j j£, Xj_i, 16fc] C G^- j£, x, /c/n] and 

[xj_i, Yj] + AS„ C Xj_i +%/[i,x, fc/n]({0} x B„_ i+ i) + AS„ 

C x + (/i 5 f / [t,x,fc/n] + A)S„ C x + 7 £„ C W (by (10)). 

This allows us to find Uj G Gj. [£, Xi_i, 16fc] C G^. [£, x, fc/n] such that 

*(a; + /i(wi,«2,...,i;i,0,0,...,0)) < + ^. (17) 

for i = 2, 3, . . . ,n. Choosing u = «2, . . . , u n ) G Gy[£, x, fc/n], we obtain (7) by summing the inequali- 
ties in (8) and (17) over i = 2, . . . , n. □ 

Now set D := x + ^B n C U. Let w/.k be a modulus of continuity for x i— > /(£, x) on if := L> + n,B n 
for all £ G [0, T] (see Condition (C 2 )). Then is also a modulus of continuity of K 3 x i— » / £ (£, x) for 
all £ G [0,T] and e > 0. The following claim parallels Claim 4.2 from [16]: 

Claim 4.2. Let (£,x,fc) G [0, T] x D x N and v G G|[£,x,fc/n]. Then ||u - / £ (£,x)|| < uj ftK {n/k) + 1/k. 
Proof. Using the Caratheodory Theorem and the definition of G|[£, x, k/n], we can write 



v = A + U 



2 
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Vi,j - x\\ <n/k a itj E [0, 1] Vi, j; 

3=1 



dij — 1 Vi; and ||A|| < 1/fc. 



In particular, x E K and yij G if for all i and j. This gives 



\vi - f e ,i(t,x)\ < Vi 




for i = 1, 2, . . . , n. The claim follows by applying the supremum norm. 



□ 



Next define 6(D) := 1 + c\ + nc 2 + c 2 max{||w|| : v E D}. It follows that 



G £ f [t,x,k/n] C {ci + c 2 (||x|| + n/k)}B n C 6(D)B n V t G [0, T], x E D, k E N. (18) 



Theorem 1 now follows from a slight variant of the argument in [16] which we include as Appendix B. 
4.2 Proof of Theorem 2 

We first make some general observations that relate our tangential and Hamiltonian conditions for strong 
invariance for an arbitrary multifunction G : R™ =$ R™. We let Hq denote the (upper) Hamiltonian for 
G (see Section 3.1). We also set D° := {p G R" : (p, d) < Vd E D} for each subset D C R"; i.e., L>° is 
the poZar se£ for D. Note that if G (z, £>) < holds if and only if G(x) C D°. 

Lemma 4.1. Let G : R™ =4 R™ and F be a closed subset of R™. (i) If G(x) C Tf (x) for all x E R, 
then Ha(x, Nft (x)) < for all x E R. (ii) If G is closed, convex, and nonempty valued and lower 
semicontinuous, and if Hq(x, N^(x)) < for all x E R, then G(x) C T^(x) for all x E R. 

Proof. Since T^(x) C (N^(x))° for all x G i? (cf. [9, Exercise 11.7.1(d)] or Appendix A), the assumptions 
of (i) imply G(x) C (N^(x))° for all x E R. This establishes part (i). To prove part (ii), first note that its 
hypotheses imply G(x) C (A- (x))° for all x E R. Let iVj£ and denote the Clarke normal and tangent 
cones, respectively (cf. Appendix A for the relevant definitions). We claim that G(x) C (N^(x))° for 
all x E R. To verify this claim, fix x G R and v E G(x). By Theorem 6 in Appendix A, we can 
find a continuous selection s : R™ — ► R" of G for which s(x) = v. Therefore, if R 3 Xj — ► x and 
N^(xi) 3 d — > C G R™, then s(xj) G G(xi) implies (s(xj), Ci) < for all i. Passing to the limit as i — > oo 
gives (w,C) < 0. Therefore, the characterization (27) for in Appendix A gives v E (N^(x))°, which 
proves the claim. It follows that G(x) C (N%(x))° = T£(x) C Tf (x) for all x E R, which proves (ii). □ 

Returning to the proof of Theorem 2, assume F(x,A) C Tg(x) for all x E S. We show that (F, 5) 
is strongly invariant by extending an argument from the appendix of [1] to our more general situation 
where the dynamics F may be discontinuous. Let M and N be any compact subsets of R™ contained in 
O such that M C int(iV). Then E := Sn N is a closed subset of R™. Let : R" -> R be any G°° function 
that is identically 1 on M, strictly positive on int(TV), and zero elsewhere. Extend F to R™ by defining 
F(x, A) = {0} outside O and define the multifunction F» : R" x A =4 R™ by F»(x, a) = <£(x) F(x, a). As 
observed in [1, Lemma A. 4, p. 1696], we have: 

Lemma 4.2. For each x E E, either F*(x, A) = {0} or Tf (x) = Tf (x). 

Therefore, since Tj?(x) is a cone for all x G F, we get F*(x, .A) C T§(x) for all x E E. Applying 
Lemma 4.1(i) with G(x) = F"(x, A) and R — E gives (x, Aj'(x)) < for all x E E. Since the multi- 
function x i ► F"(x, A) also satisfies (£/'), Corollary 3.1 implies (F",F) is strongly invariant. Therefore, 
if T > and y : [0, T] — ► O is any trajectory of F such that y(0) E S, and if we specialize the preceding 
argument to the compact set M = {y(t) : < t < T}, then y E Traj(F") (because = 1 on M) and 
y(0) E E, so y remains in E C 5. This shows (F, S 1 ) is strongly invariant. 
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Conversely, assume that (F, S) is strongly invariant. We need to show that F(x, A) C Tg(x) for all 
x G S. To this end, lixieS and a £ A. Since SCO and O C R™ is open, we can find /i > such that 
M := x + \iB n C C Choose any compact set TV C O such that M C int(iV), and let : R™ ->■ R be any 
C°° function satisfying the requirements above; i.e., 4> = 1 on M, > on int(A), and <fi = outside 
int(A). Define F" as before. Then x i— ► F$(x,a) is closed and convex valued and lower semicontinuous. 

Let E = S l~l N as before, C € N§(x), and w G F^x, a) be such that 

^ F «(,a)(S,C) = (C^)- (19) 

Such a v exists because the sets -Dj(x) are compact. Reapplying Michael's Selection Theorem (Theorem 
6 in Appendix A) provides a continuous selection s : R™ — > R" of a; i— ► F"(x,a) such that s(x) = v. 
The characterization (26) of the proximal normal cone in Appendix A gives a constant a > such 
that (C,x' — x) < a\\x' — x|| 2 for all x' G E. Let z be a C 1 local solution of the initial value problem 
i = s(z), z(0) = x, which we can assume remains in M. Since (f> = 1 on M, z is also a trajectory of 
F. Since we are assuming (F, S) is strongly invariant, z also stays in 5. Since S fl M C £7, we get 
(C, z(i) — x) < cr||z(t) — x|| 2 for small i > 0. Since i(0) = w, dividing by t > and letting t — > + gives 

(C,«> < t7jimt||(«(t)-x)/t|| 2 = 0. (20) 

Since </>(x) = 1, and since C G Ng(x) and o G A were arbitrary, (19)-(20) give Hp(x, N^(x)) < 0. 
The preceding argument applies to any x G int(A) n S, by choosing to be 1 near x and compactly 
supported on N, so H F (x, N%(x)) < for all x G int(iV) n 5". If we now fix such a function 6, then we 
have 4>(x)F(x, A) =: = {0} for all x ^ int(iV), so H Fi (x,N%(x)) < for all x G AT n S = E. 

Applying Lemma 4.1(h) with G(x) = F^(x,a), R = E, and any a G A gives (f>(x)F(x, A) C T^(x) 
for all x £ E. It x £ E f~] int(A) and F* 1 (x, A) = {0}, then F(x, A) = {0}, because <p{x) > 0; while if 
x e ED int(A) and F^x, A) ^ {0}, then Lemma 4.2 and the fact that </»(x) > give F(x, A) C T|(x). 
Therefore, F(x, A) C T§{x) for all x G E n int(A) = 5 n int(A). Now we fix x G S* and /i > such that 
M := x + iiB n C O and apply the preceding argument to all possible compact sets N C O containing 
M in their interiors. Since O is the union of the interiors of such sets AT, the preceding argument gives 
F(x, A) C Tf (x) for all x G 5 n O = 5, as desired. This proves Theorem 2. 

5 Infinitesimal Characterizations of Monotonicity 

5.1 Background and Statement of Results 

In this section, we use Theorem 2 to prove new characterizations of monotonicity for control systems 
that may be non-Lipschitz in the state. Monotone control systems were introduced by Angcli and 
Sontag in [1] and have since been applied extensively in systems biology (cf. [1, 2, 3]). To simplify our 
exposition, we only consider monotone control systems evolving on Euclidean space with input values 
in U := B m but our results can be adapted to systems whose inputs are valued in any ordered Banach 
space. The relevant definitions are as follows. We are given two closed cones K C R" and K u C R m 
(called positivity cones) which we assume are convex, nonempty, and pointed (i.e., K n (—A) = {0} and 
K u n (-K u ) = {0}). We define orders on R" and M m as follows: x x X x 2 G R" if and only if x x - x 2 G A, 
and u\ >z U2 G M m if and only if u\ — G K u . The ordering on U induces an order on the set of controls 
Woo := {measurable a : [0, oo) — > U} as follows: a\ >z ct2 G U^c if and only if ai(t) — a2(t) G K u for a. a. 
t > 0. We set U [2] = {(u u u 2 ) £U x U : m h u 2 } and U™ = {(cti, a 2 ) G Woo x Moo ■ ol\ >z a 2 }. 
Our main object of study in this section is the controlled dynamic 

n 

x G G(x, u) := TT gi(x, u)Di(x), ueU, xeX (21) 

i=l 

evolving on an open set X C R" in which each gi : X x U — > R is locally Lipschitz, (i?4) is satisfied, 
and the ZVs take all their values in some compact set DCl. The dynamic can be viewed as a locally 
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Lipschitz dynamic with non-Lipschitz disturbances Di acting on its individual components. Since we are 
assuming G Di{x) for all i and x, the dynamic (21) is clearly weakly invariant for any state constraint 
set SCI, since it allows all constant trajectories. However, since (21) is not necessarily Lipschitz in the 
state variable, it may not be strongly invariant for some state constraints. Following [1], we also assume 
there is a closed set A C X that is the closure of its interior such that all trajectories of 

n 

G v {x,u) = Y[g l (x,u)V, ueU, xeX (22) 
i=i 

starting in X remain in X. We set V = int(X). Since G-p is locally Lipschitz in x and 2? C R 
is compact, the strong invariance of (Gp,A) can be checked using standard tangent cone conditions 
(cf. [1, Appendix A]). On the other hand, because we allow non-Lipschitz D^s, (21) is not in general 
tractable by the standard strong invariance characterizations. We also define G^ : M 2rl x -t M 2rl by 
G^(x,u) = G(xi,ui) x G(x 2 ,U2); i.e., two "stacked" copies of G with ordered inputs. 

Again following [1] , we also consider more general orders given by arbitrary closed sets rcixlas 
follows: We say that x± y x 2 G X provided (x\,X2) G F. This includes the special case of state spaces 
ordered by positivity cones K by choosing F = {(x\,X2) G X x X : x\ — x 2 G K}. With the order on 1™ 
expressed this way, we set r o = V n (V x V) . We always assume the following approximation property, 
which parallels the approximation requirement in [1]: 

(A) For each £ G bd(r), T > 0, and y G Traj T (G [21 , £), there exist T G (0,T] and two sequences 
r o 3 £fc — > £ and Traj T , (G' 2 ' , 3 yk — > y uniformly on [0, T'\ as k — > oo. 

This agrees with the approximation condition posited in [1, p. 1686] for locally Lipschitz dynamics 
G, since in that case the convergence of y^ to y follows from continuous dependence on initial values. 
Condition (A) is satisfied if for example (i) X is convex (or, even more generally, strictly star-shaped 
with respect to some interior point £ G V) and (ii) there exists a neighborhood N of bd(A x X) such 
that x i — ► G^ (x, U^) is Lipschitz on M. In this case, the existence of a sequence r o 3 £fc — > £ in (A) 
for each £ G bd(A x X) follows from (i) and the argument from [1, p. 1686]. Given y G Traj(Gl 2 ', £), the 
existence of trajectories Traj(Gl 2 l , £&) 3 yt — * y uniformly in (A) then follows because near £, we can 
write Gl 2 l(x, a(t)) = J(t,x,B n ) for some Lipschitz parametrization J and the input a G for y (see 
[4, Chapter 9]), so y = J(t,y,/3) for some input (3 and small times, and then we can apply continuous 
dependence on initial conditions to the dynamics J. It suffices to check (A) for £ G hd(X x X), since 
r\bd(A x A) C r o . On the other hand, we do not need to assume Lipschitzness of (21) in a neighborhood 
of bd(r o ) as was needed in [1]. We call (21) monotone provided: 

(M) If ai y ot2 G Woo, x\ y X2 G X, T > 0, (f>i G Traj T (G, ai,x\), and (j>2 G Traj T (G, a2,x 2 ), then 
Mt) h <fa{t) for all t G [0,T\. 

In other words, (21) is monotone provided its flow map preserves the orders on its inputs and initial 
states. Condition (M) differs slightly from the definition of monotonicity in [1] because our non-Lipschitz 
dynamics generally admit multiple solution trajectories for some choices of inputs and initial states. Note 
that G is monotone if and only if (G[ 2 l,r) is strongly invariant. Moreover, G^ satisfies the hypotheses 
of Theorem 2 (see Example 2.2). For state spaces ordered by positivity cones K, our main result is: 

Theorem 3. The dynamics (21) is monotone if and only if the following condition is satisfied for all 
&,&eV: (Ziyb,u 1 yu2)^G(Z 1 ,u 1 )-G(&,U2)_\TB(Z 1 -&). 

It is easy to check that the tangent cone condition in Theorem 3 is equivalent to the following for 
each G V: - & G bd(K), Ul h u 2 ) => - G(£ 2 ,u 2 ) Q T§(^ - £ 2 ). This is because 

Tk(x) = R" for all x G int(K). For orders induced by closed sets r C X x A, we also prove: 

Theorem 4. The dynamics (21) is monotone if and only if the following condition is satisfied for all 
£i)£2 G V: (£i y £2, ui y U2) =>■ Gl 2 l(£,u) C T-p(0- 
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Our theorems extend the monotonicity characterizations in [1] by allowing non-Lipschitz dynamics. 
The dynamics in [1] take the form G(x, u) — {g(x, u) : u E U} where g is continuous and locally Lipschitz 
in x locally uniformly in u. Since we allow each factor x ^ Di{x) in our dynamics (21) to be set valued 
and non-Lipschitz, our dynamics may be non-Lipschitz in the state. Therefore, the strong invariance 
theory from [9, Chapter 4] used in [1] no longer applies. Instead, we prove Theorems 3-4 using our new 
strong invariance theory from Section 4. 

5.2 Proof of Theorems 3 and 4 

The following lemmas parallel the corresponding lemmas in [1, Section III]. 
Lemma 5.1. (G, V) is strongly invariant. 

Proof. Let x E V, a E Uoo, T > 0, and y E Traj T (G, a, x). We show y(t) E V for all t E [0, T]. Note that 
there exist measurable functions fa : [0, T] — ► K with f3i(t) E Di(y(tj) for all i and a. a. t E [0,T] such 
that y is the unique solution of 

n 

x = Y[gi(x,a(t))0i(t) (23) 

i=l 

on [0, T] starting at x. This follows from the Filippov Selection Theorem as in Example 2.2. For each 
x E V, let [0, T] 3 ( h z(t, x) be the solution of (23) starting at x. Notice that z(-,x) is a trajectory 
for Gx>(x,a(t)) from (22) for each x E V and therefore remains in X by assumption. Fix t a E [0,T]. 
Viewing (23) as a system with control inputs a and (3i and arguing as in [1, Lemma III. 6] (see [20, 
Lemma 4.3.8]) shows that the image .F(V) of the final point map V3ih '■= z (toi x) E X contains 
an open neighborhood W of T(x). By assumption, W C J-(V) C X, so W C int(X) = V. Therefore, 
y{t a ) — J-{x) E W C V. Since t a E [0, T] was arbitrary, y stays in V. This proves the lemma. □ 

Recall that G is monotone if and only if (G^ , V) is strongly invariant. We next assume that the order 
on K" is expressed in terms of a closed set rcixlas above. 

Lemma 5.2. The dynamics G is monotone if and only if (G[ 2 l,r o ) is strongly invariant. 

Proof. If G is monotone, then (G^ , V) is strongly invariant. By the previous lemma, (G^ , V x V) is also 
strongly invariant, so the sufficiency follows because r o = V n (V x V). Conversely, assume (G^ 2 ', T ) is 
strongly invariant. We show (G^jT) is strongly invariant using our approximation hypothesis (A). Let 
T > and z : [0, T] — > R" x R" be any trajectory of G^ starting at a point in T, and define 

i d = sup{t G [0, T] : z{s) ETVsE [0, t}}. (24) 

We need to show that t = T. Suppose the contrary, so t < T. Notice that z(s) E T for all s E [0, t]. 
In particular, ^ := z(t) E bd(r). We apply (A) to the trajectory [0, T - t] 3 t ^ y(t) := z(t + 1) of 
G [21 starting at f. This gives T' G (0, T - f| and sequences r o 9 £ fc -> C and Traj T , (G [2] , ^ fe ) 3 y k ^ y 
uniformly on [0, T'\. By hypothesis, the y^s remain in r o C T, so their uniform limit also remains in the 
closed set r on [0,T']. Therefore, z remains in T on [0, i + T'], which contradicts our definition of i in 
(24). This establishes that (G^T) is strongly invariant and completes the proof of the lemma. □ 

We next relate the tangential conditions for G and G^ from Theorems 3-4. 

Lemma 5.3. For any £ E (£1,^2) S r o and u = (ui,u 2 ) E U^ 2 \ the following three conditions are 
equivalent: (a) G(£i,«i) - G(^u 2 ) C Tgfa - £ 2 ), (b) G^(^u) C T r S o (e), and (c) G^u) C Tj? (£). 

Proof. First fix £ = (£i,£ 2 ) G r o , u = (ui,u 2 ) E U [2] and values v l E A(£i) and w i G A(6)- Set 

(n n \ 

^.giCx^ui)^,]^ ^(0:2,^2)^ I . (25) 
i=l i=l / 
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Then / is locally Lipschitz in x although its trajectories are not necessarily trajectories of G' 2 '. The 
lemma follows if the following are equivalent for all £ £ T and u £ : 

(a') /i(&,ui) - / 2 (6,«2) G - £ 2 ), (6') /(£,«) e T£(0, and (c') /(£,«) e Tj?(£). 

Since (25) is locally Lipschitz in x, this equivalence follows from the proof of Lemma III. 9 in [1]. □ 

We can now prove our two monotonicity theorems. Assume (21) is monotone, (£1,^2) £ F , and 
u° = (u°, u%) £ . By Lemma 5.2, (G' 2 ' , r o ) is strongly invariant. Applying Theorem 2 from Section 4 
to the dynamics £ ^ G [2] (£, u°) in dimension 2n with O = V x V and S = T gives G [2] (£, u°) £ Tj? (£) for 
all £ £ r o . The tangential conditions from both theorems therefore follow from Lemma 5.3. Conversely, 
assume either of these tangential conditions. By Lemma 5.2, monotonicity of (21) follows once we show 
that (G^l , r o ) is strongly invariant, but this follows from Theorem 2 applied to G^ 2 ' because (b) of 
Lemma 5.3 is satisfied for all u £ A := and £ £ T . This concludes the proof of our theorems. 

Remark 5.4. The tangential conditions from our monotonicity theorems remain sufficient for mono- 
tonicity if we drop the convex valuedness and lower semicontinuity assumptions on the IV s and keep 
all our other assumptions the same (by the same proof). Moreover, the requirement that £ Di(x) for 
all i and x can be relaxed to requiring an increasing sequence N\ C N2 C . . . C N)~ C ... of compact 
subsets of V x V such that (i) V x V = Ufc7V fc and (ii) £ A(#i) x A (^2) for all i £ {1, 2, ... , n}, all 
x = (xi,X2) £ bd(r o n Nk), and all fc (see Section 3.2). 
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A Background in Nonsmooth Analysis 

In this appendix, we review the necessary background in nonsmooth analysis; see [9] for a more complete 
treatment and Section 2 above for the relevant notation. Let S C M" be closed and x G S. A vector 
( G K" is called a proximal normal of S at x provided there exists a constant a = a((, x) > such that 

(C,x'-x) <<t\\x'-x\\ 2 . (26) 

for all x' G S. The set of all proximal normal vectors of S at x is denoted by Ng(x). The following local 
characterization of Ng also holds: For any 5 > 0, C £ Ng(x) if and only if there exists a = cr((,x) > 
such that (26) holds for all x 1 G S Pi (x + 5B n ). We also define the Clarke tangent cone T$ and the 
Bouligand (a.k.a. contingent) tangent cone Tg to subsets S C W 1 as follows. We say that v G Tg{x) 
provided for each sequence x, G S converging to x and each sequence U > decreasing to 0, there exists 
a sequence Vi — > v such that x t + Uv, G S for all i. In particular, if S = {0}, then Tg(0) = {0}. The 
Bouligand tangent cone to S is defined by 

T§{x) := {q G R" : 3U G (0, oo) and S 3 x t -> x s.t. U j and (l/U){ Xl - x) -> q} Vx G S. 

Then Tg(x) C (Ng(x))° for all x G S and all closed sets S C R™, where the superscript o denotes the 
polar set; i.e., D° := {pel": (p, d) < OVc? G D} for each D C M™. The Clarke and Bouligand tangent 
cones can differ for some sets S but are known to agree when S is a closed convex subset of R™. We also 
use the Clarke normal cone which can be characterized for any closed subset R C M™ as follows: 

N%(x) = co { lim d : & G N^{ Xi ),R3 Xl -» x] (27) 

Then (NC(x))° = T%{x) C T%{x) for all x G R. 
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Next assume / : R™ — > (—00,00] is lower semicontinuous and x E domain(/) := {x' : f(x') < 00}. 
Then £ E R™ is called a proximal subgradient for f at x provided there exist cr > and 77 > such that 

f(x')>.f(x) + ((,x>-x)-<j\\x>-x\\ 2 (28) 

for all x' E x + r/B„. The (possibly empty) set of all proximal subgradients for / at x is denoted by 
dpf(x). When / is the characteristic function of a closed set S C R" (i.e., /(x) = if x E 5 and 
/(x) = 1 otherwise), it follows from the local characterization of Ng and (28) that dpf(x) — Ng(x) for 
all x E bd(5) and dpf(x) = {0} otherwise. 

We next state a version of the Clarke-Ledyaev Mean Value Inequality (cf. [9, p. 117] for its proof). 
Let [x, Y] denote the closed convex hull of x E R™ and Y C R™. 

Theorem 5. Assume x E R™ 7 7 C M" is compact and convex, and \P : R™ — ► R zs /ower semicontinuous. 
Then for any 5 < min ye y &(y) — Vt'(x) and A > 0, there exist z E [x, F] + XB n and ( E 9p^(^) such that 
S < (£, y — x) for all y <EY . 

A function s : R n -> R" is called a selection of F : R™ =4 R" provided s(x) E F(x) for all x E R™. 
The following result is known as Michael's Selection Theorem (see [5, Chapter 8] or [19, Corollary 5.59]). 

Theorem 6. Let F : R" =4 R™ be lower semicontinuous and closed, convex, and nonempty valued. Let 
x E R" and v e F(x). Then there exists a continuous selection s of F for which s(x) = v. 

The following is a variant of the well known "compactness of trajectories" lemma which we use in 
Appendix B. Its proof is a special case of the proof of [9, Theorem IV. 1.11]. 

Lemma A.l. Let x E R™, T > 0, / E C[0, T] be also continuous in t, and yu : [0, T] — > R™ be a sequence 
of uniformly bounded absolutely continuous functions satisfying j/fe(0) = x for all k. Assume 

Vk{t) E f(T k (t),y k (t) + r k (t)) + S k (t)B n (29) 

for a. a. t E [0, T] and all k, where 5 k : [0, T] —> [0, 00) is a sequence of measurable functions that converges 
to in L 2 [0,T] as k — > 00, r k : [0, T] —> R is a sequence of measurable functions converging uniformly 
to as k — > 00, and r k : [0, T] — ► [0, 00) is a sequence of measurable functions converging uniformly to 
r(t) = t as k — > 00. Then there exists a trajectory y of y = f(t, y), y(0) = x such that a subsequence of 
y k converges to y uniformly on [0,T]. 

We apply Lemma A.l to continuous mollifications of our feedback realizations / E C[0, T] defined as 
follows. We first define the standard mollifier 




where the constant C > is chosen so that J R ri(s)ds — 1. For each e > and t E R, set r] e (t) := n(t/e)/e. 
Notice that J M r) e (t)dt = 1 for all e > 0. Define the following mollifications of / E C[0, T}: 

f £ (t,x) := [ f(s,x)r le (t-s)ds (30) 

JR 

with the convention that f(s, x) = for all s ^ [0, T}. Then f e E C[0, T] and (t, x) 1— > / e (i, x) is continuous 
for all e > 0. (See [13, Appendix C] for the theory of convolutions and mollifiers.) We apply Lemma A.l 
to a sequence f £ (k) °f naollifications with e{k) > converging to zero using ideas from the usual proof 
that / e (fe)(-,x) — > /(-,x) in L 1 ^, T] for each x and / E C[0,T] as fc — > 00. 

Remark A. 2. If r k (t) = t for all k in Lemma A.l, then the conclusions of the lemma remain true even 
if the t-continuity hypothesis on / E C[0,T] is omitted. This follows from the proof of the compactness 
of trajectories lemma in [9] and is applied to (46) below. 
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B Additional Proofs 

In this appendix, we summarize the argument from [16] needed to complete our proof of Theorem 1. We 
continue to use the notation from Section 4.1 and we set 

T ' : = min { T '32^)} md hk --=32kW) < 31 > 
for all k G N. Choose N > 2 such that 

D + h k S(D)B n C x + ^B n \fk>N. (32) 

By the choices of 7 G (0, 1) and 5(D), 

0</ ^ 32k 9f[ t,x,k/n ] ^6[0,n,»€A*€N. (33) 

Next we define c(fc) = Ceiling (T'/hk); i.e., c(fc) is the smallest integer > T' /hk- For each k > N, we 
then define a partition 7r(fc) : = to,fc < ti,k < ■ ■ ■ < i c (fc),fc = by setting = t{-\^ + hk for 
i = 1,2, . . . ,c(fc) - 1. 

We next define sequences Xo,k, ^l.fc, £2,fc, ■ • • , £ c (fc),fc for fc > TV as follows. We set £o,fc = £ and 

£i,fc = S + (ti ;fe - io,fc)u ,fe, (34) 

where w = v ,fc G [0, x, k/n] satisfies the requirement from Claim 4.1 for the pair (t,x) = (0,x) and 
h = h k . By (18) and (34), we get 

\\xi,k-x\\ < hkS(D) = ^1 (35) 
so X\ t k G x + ^B n = D. If c(fc) > 2, then we set 

X2,k = X\ t k + (t2,k - ti,k)vi,k, (36) 

where v\ t k G G E f\t\,k, k/n] satisfies the requirement from Claim 4.1 for the pair (t,x) — {t\ t k,x\ t k) 
and h = hk- Then (18) and (36) give \\x2,k — £i,fc|| < hkS(D) = so 

ll^-*ll<^. (37) 

by (35). By (37), x-i.k G D. We now repeat this process except with x^.k £ D replacing x\ t k- Proceeding 
inductively gives sequences v^k G G £ j[ti y k,Xi y k,k/n} and x iy k that satisfy 

Xi+i,k = Xi,k + {U+i,k - ti,k)vi,k (38) 

for each index i — 0,1, ... , c(k) — 1. The wife's are chosen using Claim 4.1 with the choices h = hk and 
(t,x) — (ti t k,Xi,k) for all i and k. The choices of T" and k > 2 and (38) give 

11 l,fe 11 - 32fc " \h k J 4fc " 2 

for all i and /c. It follows that the sequences x iy k lie in D. 

For each k > N, we then choose to be the unique polygonal arc satisfying ^(^(O) = x and 

**(*)(*) = fe{Tk{t),X <k ){t)+r k {t))+Zk{Tk{t)) (39) 

for all t G [0, T"] \ n(k), where Tk(t) is the partition point U k G 7r(fc) immediately preceding i for each 

te[o,n 

Zfc(*i,k) :=u»,fc- fe(U,k,x^ k )(U,k)) Vi.fc (40) 
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the Vi : k G G £ f[U t k, ^7r(fe)(*i,fe)) k/n] satisfy the conclusions from Claim 4.1 for the pairs (t,x) = (U tk ,Xi tk ) 
and h — hk, and 

r k {t) := x n{k) (T k (t)) - x w{k) (t) Vi G [0,T'l Vfc. (41) 

Then av^) is the polygonal arc connecting the points 27^ for i = 0, 1, 2, ... , c(k) so Xi ik = a^ffe) (^,fe)- 

Since (t, x) *—> f e (t, x) is continuous, we can use Claim 4.2 and the forms of z k and r k in (40)-(41) to 
check that (39) satisfies the requirements of our compactness of trajectories lemma with y k = 2^0) (see 
[16]), so we can find a subsequence of x^k) that converges uniformly to a trajectory y £ of y = f s (t,y), 
y(0) = x. By possibly passing to a subsequence without relabelling, we can assume that x w r k -\ — > y e 
uniformly on [0,T']. Since t i+1 , k - t i;fe G [0, h k ] and x i+1 , k = x hk + (t i+ \ tk - t iik )v iik G D for all 
i = 0, 1, . . . , c(k) — 1 and k > N, conditions (32) and (33) and Claim 4.1 give 

tffofc) - *(zi-i lfc ) < y (42) 
for i = 1, 2, . . . , c(fc). Summing the inequalities (42) over i and noting that h k < 7 gives 

*(*i,fc) < *(x) + £^i < vE-( 5 ) + I(T' + 7 ) (43) 
for all i and A; . Since x n ^) ( T fe(*)) — x i,fc for ah i G [t*,fe, *»+i,fc) and all z and k, (43) gives 

*(^ (fe) (r fe (i))) < + i(T' + 7) (44) 

for all i G [0,T']. Since |r fe (i) - i| < h k -> as k -> +00 for all t G [0,T'], a; 7r(fe) (r fe (i)) -> y e (t) for all 
i G [0, T"] as fc — > +00. Moreover, ^ is lower semicontinuous, so it follows from (44) that 

*(lfe(*)) < (45) 

for all t G [0,T']. 

Now let yi/i : [0, T'] — > R™ be the trajectory obtained by the preceding argument with the choice 
e = 1/i for each « G N. Note that y\/i{t) G D for all i and t because each of the polygonal arcs x^^) 
constructed above joins points in D and D is closed and convex. Moreover, 

yi/i(t) = f(t,yi/i(t)) + [fi/i(t, yi/i (t))- f(t, yi/i (t))] (46) 

for all i and almost all t G [0, T']. One can check (see [16]) that the yi/j's are also equicontinuous, so 
we can assume (by passing to a subsequence) there is a continuous function y : [0, T'] — ► D such that 
Vi/i ~^ V uniformly on [0, T'] (by the Ascoli-Arzela lemma). One can show (cf. [16, Claim 4.3]) that 

/i/i(*,»i/i(*))-/(*,yi/iW)-0 in i 2 [0,T'] (47) 

as i — > 00. The proof of (47) is based on the standard proof that x) — f(-, x) — > in L 1 ^, T'] for each 

a; (see for example [13, pp. 630-1]) and property (C2) from the definition of C[0,T]. It therefore follows 
from Remark A. 2 and the form of the dynamics (46) that a subsequence of t/i/j converges uniformly 
to some trajectory of / on [0, T"]. This must be the aforementioned function y, which is therefore a 
trajectory of /. Again using the lower semicontinuity of (45) applied along the sequence e = 1/i gives 

*(l/(t)) < liminf *(!/!/((*)) < *(*) < 

z — ► OO 

for all i G [0,T']. In particular, y G Traj T ,(F,a:) and y : [0,T'] -> 5. 

Finally, we prove the strong invariance assertion of the theorem. We assume i„ £ 5, T > 0, and 
z G Traj T (F,a; ). Set 

t := sup{i > : V(z(s)) < OVs G [0,t]}. (48) 
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We next show that i — T (by contradiction), which would imply that z remains in S on [0,T] and 
establish the theorem. Suppose t <T. The lower semicontinuity of \& gives ^>(z(t )) < 0. The definition 
of t implies that z(-) £ S on [0, i] and in particular x := z(t) £ bd(<S) C U. Next we let / £ Cp([0, T— t\,x) 
satisfy the requirement (£/') for F and the following trajectory for F: 

[0,T -t\3t^y(t) := z(t + t). (49) 

By definition, we can then find 7 G (0,1) satisfying f(t,x) £ ccone{F(x)} for a. a. t £ [0,7) and all 
i£i + "fB n . By reducing 7 > without relabelling, we can assume x, + jB n C U. 

By uniqueness of solutions of the initial value problem y = f(t,y), y(0) = z(t) on [0, T — 1\, the above 
argument applied to / and x = z(t) gives t £ (0, T — i ) such that 

V(z(t + t)) < We [0,t\. (50) 

Here we use the fact that the trajectory for / on [0, T'\ starting at x. (and valued in <S) that we constructed 
above (where we can assume T" < 7) can be extended to a new trajectory for / defined on all of [0, T — 1\ 
by the linear growth assumption (C3) on / and so coincides with (49) by our uniqueness assumption in 
([/'). We can therefore set t — T' . Since z remains in <S on [0,t\, we conclude from (50) that z remains 
in S on [0, i + 1\. This contradicts the definition (48) of t and proves Theorem 1. 

Remark B.l. The preceding proof provides a constructive approach to finding viable trajectories for our 
Caratheodory feedback realizations / that remain in S. As suggested by [14], an alternative but highly 
nonconstructive proof of Theorem 1 would proceed as follows. Let x(t) be any trajectory of F starting in 
S. By Condition (U 1 ), x(t) admits a feedback realization / £ C[0,T], and our Hamiltonian assumption 
gives ((f(t,x),0),q) < for all q £ N* im (x, = {(£,-1) : £ £ d P ^{x)}, almost all t > 0, and all 

x £ U, where epi (W) := {(x,r) : r > ^(x)} is the epigraph of It is not hard to deduce from this 
(cf. [9]) that (f(t, x),0) £ T£ im (x,r) for almost all t > 0, all x £ U, and all (x,r) £ epi(*), where 
T B denotes the Bouligand tangent cone. Applying the measurable viability theorem (see [15, Section 4]) 
to the dynamics F(t,x) — {(f(t, x), 0)} provides a trajectory t 1— ► (4>(t), ^ r (x(0))) for x = f(t,x), y = 
starting at (x(0),^(x(0) j) that stays in epi(*). This requires / to be modified outside U in the usual 
way. Hence, ^(<p(t)) < ^(x(0)) < for all t. By the uniqueness part of Condition (U'), 4>(t) = x(t) so 
x(t) stays in S. Unfortunately, the preceding alternative argument is highly nonconstructive, since the 
proof of the measurable viability theorem relies on Zorn's Lemma to construct the trajectory <fi(t). One 
natural question which should be considered is how our Euler constructions from our proof could be used 
to build numerical schemes for approximating viable trajectories for Caratheodory dynamics. This type 
of result could be useful in physical applications. This question will be addressed by the author in future 
research. 
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